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ABSTRACT 


This thesis studies the employment of an information flow reduction 
and sorting system. The system is designed to reduce the amount of 
information gathered by a collection system to a rate that a user of that 
information can accept. The thesis demonstrates the benefits of trait-based 
analysis of information as a method of screening desired information from 
undesired. These systems increase the quality of the information reaching | 
the user while adding a delay to achieve the screening process. A method of 
networking the screening devices is discussed. A sorting system is added to 
the screening process to demonstrate its ability to increase the speed of desired 
information through the system. The models are illustrated through 
numerical examples. The analysis provides the user of these systems with an 
understanding of their design, employment, benefits, costs and calibration 


requirements. 
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EXECUTIVE SUMMARY 


Information collection systems gather information from the 
environment and deliver it to a decision maker. The decision maker’s goal is 
to evaluate this information and then act upon it based upon what he/she 
has learned. The decision maker has conflicting requirements for the 
collection system: high quality information and high volume of information. 
These requirements conflict because the decision maker’s enemy desires to 
counter the collection efforts. The enemy obscures vital information from 
the decision maker by mixing it with non-vital information. This decreases 
the quality of the collected information. The enemy also hides vital 
information through various means, allowing the collection system to obtain 
only a small portion of what the decision maker desires. To obtain the 
desired amount of vital information, the decision maker is forced to collect 
an increasingly larger amount of raw information. Often this level exceeds 
the decision maker’s ability to analyze all of the collected items. The role of 
the decision maker, therefore, is to collect as much of the available 
information as possible, remove the non-vital information and make the 
correct decision based upon the obtained vital information in an acceptable 
amount of time. It is to the decision maker’s advantage to possess an 
automated system which reduces a high volume of collected information to a 
useable level while increasing the overall quality. 

This thesis studies the use of a trait-based information reduction 
system designed to achieve the flow reduction and quality improvement 
desired by the decision maker. The reduction system examines traits of an 
arriving piece of information to determine if those traits are similar to those 
of vital information. The system forms acceptance/rejection criteria based 


upon these traits and forwards information to the decision maker at a rate the 


decision maker can accept. An example demonstrates quality level 
improvements of 150 percent for a single trait system and 170 percent for a 
two trait system. These improvements are not without cost. The costs 
associated with trait-based selection systems include infrastructure costs such 
as design and fielding as well as time delay costs associated with the trait 
analysis. The example demonstrates a 10 percent increase in analysis time for 
a single trait system and a 14 percent increase for a two trait system. A 
method of calibrating trait-based reduction systems to achieve optimal quality 
is provided. Sensitivity of these systems to changes in the information 
environment is demonstrated. 

It is desirable to reduce the time delay associated with a trait-based 
reduction system so that the decision maker receives the vital information 
sooner. This thesis also studies the benefits of adding a trait-based sorter to a 
network of trait-based selectors. The sorter is calibrated to give vital 
information priority over non-vital information. The thesis finds that a sort 
system of this type always provides a decrease in time delay of vital 
information provided to the decision maker. The example demonstrates that 
a sorter added to the single trait reduction system decreased the overall delay 
by 15 percent. A method for obtaining the optimal decrease in delay is 
provided as well as a discussion of the sensitivity of the system to changes in 


the information environment. 
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L INTRODUCTION 


Information is the key ingredient in a decision making process. The 
structure of military organizations has a basis in the flow of information from 
those who collect it to those who determine the response to it. As the 
sophistication and capacity of information collection technology continues to 
grow, the amount of information presented to the decision maker increases. 
At face value this is a desired effect. However, the vast assortment of 
available facts presented to today’s decision makers can easily overwhelm 
his/her capability to absorb the material. Equally sophisticated information 
reduction and sorting systems attempt to remove the chaff from the wheat 
and present the decision maker with the most relevant facts. 

Reduction and sorting systems that deal with such issues are currently 
in use throughout the military. An example might be as warfighting oriented 
as a Combat Information Center, where sensor operators utilize their 
collection systems to ascertain the battle environment. The task of these 
operators is to present the best picture to the Tactical Action Officer (TAQ) for 
display as well as to provide assessments on particular portions of that 
information. The TAO must then make tactical decisions based upon the 
presented information. National level agencies collect information in a 
similar fashion and with the same underlying purpose: to provide a decision 
maker with as many available and relevant facts and assessments on the issue 
under evaluation in the available time. 

Systems of this type possess inherent problems that confound the 
decision process. Received information is likely to be incomplete or 
shrouded from its absolute meaning. The cause may be due to limitations in 
the collection system. The radar holds an air contact but is it military or 
commercial? Is it friend or foe? Active denial of information as well as 


infusion of false information by the target increases the chance that the 


assessment will be incorrect. This particularly applies to military and 
national level information. Systems and procedures designed to respond to 
clarify the vague information often require the application of assets and time. 
The TAO may require the intercept of the air contact for visual identification. 
The intelligence officer may require the decryption of the enciphered 
information. This delay may allow the target to operate within the decision 
maker’s decision cycle and afford the enemy tactical or strategic advantage. 
Incorrect assessment of the information also negatively impacts the decision 
by providing a falsely high or falsely low evaluation of the threat. Allowing 
either to occur is detrimental to the decision maker’s ability to effectively 
respond. 

The objective of this thesis is to provide a model for the analysis of an 
imperfect information flow reduction and sorting system. Envision the 
system as a network of sorting and analysis devices, Figure 1, within which 
items of information are placed into priority levels according to discrete 
features of the items. The resultant stack is of items in prioritized order 
presented to a decision maker. The decision maker does not have the 
capability to review the entire stack but will review the objects from highest 
to lowest level until he must make a decision. The system is imperfect due to 
its inability to always place an object within the proper priority level. Three 
results with two types of errors can exist during the sorting of each object. 

Ls The object receives the correct priority level. No error occurred. 

Di The object receives a lower priority than it actually holds. This 
reduces the quality of the objects available to the decision maker or delays the 
time of presentation. 

3; The object receives a higher priority than it actually holds. This 
confounds the decision process by placing some chaff with the wheat. It also 
lowers the efficiency of the system. 

The problem is further defined in Chapter Hl. The models are 
developed in Chapter III using a simple reduction system as a base-case. 


Section E of Chapter III discusses the addition of a single measurement and 


selector pair. Section F links an additional pair and notes the implication of 
networking such pairs. Section G develops the sort system model as an 
enhancement to a single selector model. Chapter IV discusses the calibration 
of these devices and their sensitivity to changes in the inputs. Chapter V 


provides insight on the models and recommendations for further analysis. 
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Figure 1. Generic System Ilustration with Model Development Notation 





IL. PROBLEM DISCUSSION 


A. GENERAL SYSTEM MODELS 


Information collection systems gather information of various forms 
from the environment and pass it to a decision maker. The decision maker 
wishes to make the best decision possible with the available information in 
the available time. Often, the collection systems can collect more information 
than the decision maker can review and process. This causes a backlog in 
available information and results in a decision maker who is constantly 
analyzing information and not using his/her time to make the required 
decisions. The result is information overload. Information flow reduction 
systems seek to lessen the amount of information reaching a decision maker 


while, if possible, increasing the quality of the information available. 


1. Simple Information Flow Reduction Systems 
The simplest method of flow reduction is to require the amount of 


information passed to the decision maker to be less than a previously set 
threshold. The decision maker must set this limit based upon his/her own 
capabilities. Simple information flow reduction systems make discriminating 
decisions on which items to reject and which to retain. Three methods exist 
for achieving this type of flow reduction: 

° rejecting previously collected information, 

° reducing the amount of information gathered by the collection 

system and, 
° reshaping the environment from which information is 


obtained. 


This thesis analyzes the flow reduction that results from the use of a 
simple rejection procedure. The thesis does not examine the reduction in the 
amount of information gathered by the collection system. Section 4 contains 


a discussion regarding the reshaping of the information environment. 


De More Complex Information Flow Reduction Systems 
A more complex method of reducing the flow of information to a 


decision maker is to allow for devices between the collection system and the 
decision maker to pre-analyze the data. These pre-analyzers seek to remove 
the same level of flow as the simple reduction system. However, the pre- 
analyzers have the additional capability of examining traits of the 
information and determining acceptance and rejection based upon those 
traits. The benefit of utilizing a pre-analyzer is that it can discriminate 
between the types of information retained and rejected. 


This thesis demonstrates the benefits of one and two trait analyses. 


3; Sorting Enhanced Information Flow Reduction Systems 
An additional feature of a pre-analyzer based flow reduction system is a 


sort system. A sort system allows for the further refinement of information 
into prioritized levels. A discriminating pre-analyzer based system can place 
the more vital retained information higher in priority than less vital 
information. The decision maker then accesses the high priority items 
sooner than the low priority items and achieves a greater efficiency in quality 
information throughput. 

This thesis demonstrates the benefits of adding a sort system to a pre- 


analyzer based flow reduction system. 


thesis. 


TERMINOLOGY 


The following terminology applies throughout the remainder of this 


Good 


Bad 


Value 


Trait 


Trait Level 


Selecting 


Trashing 


Selection 


Type I Error 


Type I Error 
Stack 


when applied to information, it implies the item is 
important to the decision maker. 

when applied to information, it implies the item is 
not important to the decision maker. 

the state of being either good or bad. No further 
distinction is made on the relative importance of 
one good (bad) item to another good (bad) item. 

A feature which is used to describe an item. Traits 
depend upon the information under analysis but 
could include length, frequency, altitude, speed, 
Origin, etc.. 

A measurement returned by a pre-analyzer which 
describes a trait of the item being analyzed. 

allowing an information item to proceed further 
through the system. 

removing or rejecting an information item from 
the system. Once trashed an item is unrecoverable. 
the entire process of selecting and _ trashing 
information items. 

trashing a good information item. 

selecting a bad information item. 

location of information items just prior to the 
decision maker analyzing them but following the 
flow reduction system; i.e. the decision maker's in- 
box. 


Prioritized Stack a stack with various levels. Each level denoting a 


ranking, high to low, for the decision maker to 


obtain information items. 


© PARAMETERS IN THE ANALYSIS 


The following parameters are features of the system and the 


information environment and do not change. 


Pe 


Ps 


AcoLLECT 


Lom 


Ppa 


—= 
= 
— 


it 


lll 


lil 


Probability an item collected from the environment is 
Good. 

Probability an item collected from the environment is 
Bad 

lore. 

Arrival rate of items from the collection system. 

Service rate of the decision maker when analyzing 
information (Items / Unit Time). 

Fraction of the time the decision maker is analyzing 
information (set by the decision maker). 

Total flow rate of items to the decision maker following 


flow reduction 


+ 
Pom) Up: 


These parameters provide an understanding of the information 


environment, the capabilities of the collection system to collect from that 


environment and the abilities for the decision maker to process the available 


information. P, is the fraction of good information collected by the collection 


system. It is a function of the overall availability of good information as well 


as the biases incorporated in the collection system to collect good information. 


P, is the fraction of information collected which is not important to the 


decision maker. 


ANcottecr iS the rate at which the collection system operates 


and is assumed to be much higher than the rate at which the decision maker 


can process the flow, Upy. Because of this and because the decision maker 


must leave time to act upon the information, he/she sets a limited amount of 


time aside to analyze information and uses the remaining time to make 


decisions. This implies there is an acceptable flow rate to the decision maker, 


Xpm, at which the decision maker can analyze all available information in 


his/her allotted time and then make decisions in the remaining time. 


D. MEASURES OF EFFECTIVENESS 


The models’ measures of effectiveness [MOEs] include: 


Pom GOOD 


PryrE I ERROR 


PrRasi BAD 


Pie TT ERROR 


Ppm Goop 


Ppm BaD 


Probability an important item which has been collected 
reaches the decision maker. 

Probability an important item which has been collected 
does not reach the decision maker. 

Probability a non-important item which has been 
collected does not reach the decision maker. 

Probability a non-important item which has been 
collected reaches the decision maker. 

Fraction of the decision maker’s analysis time spent 
analyzing important items. 

Fraction of the decision maker’s analysis time spent 


analyzing non-important items. 


Define the waiting time in the system to be the delay incurred by 


passing an item through the flow reduction system and the decision maker. 


This delay begins when an item leaves the collection system and ends when 


the decision maker has analyzed the item or the item has been rejected by the 


system. 


Wom GOOD 


= Expected waiting time in the system for important items 


that reach the decision maker. 


Wren error = Expected waiting time in the system for non-important 


items that reach the decision maker. 


Expected waiting time in the system for non-important 


Werrasi BAD 


items that are rejected by the system. 


Wryrel1 error = Expected waiting time in the system for important items 


that are rejected by the system. 


These MOEs provide a number of ways of measuring the ability of the 
system to perform its function of flow reduction and quality enhancement. 
When compared across’system types, Pi coop Pies n errow Liner eeeon ee 
Prrash pap Gemonstrate the system enhancement’s ability to improve the 
quality of information flow to the decision maker. 

The decision maker first sets his/her p,,,, the fraction of the decision 
maker’s total time that is used to analyze information. Presumably, the 
decision maker would like to use much of this available time analyzing good 
items. Recall that Ppy gap is the fraction of time wasted on examining 
valueless information. When compared across system types it is desired that 
system enhancements provide an increase in Ppy coop and a corresponding 
decrease in Ppy pap: 

The MOEs describing the expected waiting time in the system provide 
an understanding of the amount of time required to achieve the quality and 
efficiency MOEs previously described. Woy coop indicates the expected waiting 
time between the time an item is collected and the time the decision maker 
can act upon that piece of information. Wy. 1p error indicates the waiting time 
for non-important items that reach the decision maker. Because all items 
which reach the decision maker will eventually be processed, it is desired that 
the delay in the system for important items be as low as possible, and lower 
than that of non-important items. Conceptually, there is no loss to the 
decision maker if he/she never gets around to examining a non-important 
item but there can be a great loss in delaying an important piece of 


information. Wryrpast pap indicates how long the pre-processing system requires 


10 


before it removes a bad item from the flow to the decision maker. Wry.) Error 
provides an indication of how long the pre-processing system requires before 
it errs and removes a good item from the flow to the decision maker. Since 
all trashing takes place prior to reaching the analytic section, these MOEs 
provide an understanding of the flow reduction system’s efficiency in 


performing its function. 
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Hl. MODEL STRUCTURE 


A. ARRIVAL PROCESS AND SERVICE TIME DISTRIBUTIONS 


Throughout the analysis of these models the precise methods by which 
the devices in the system achieve their requirements are of little import. 
Where applicable, methods are discussed to give the reader a fundamental 
understanding of the issues involved. More importantly, the models provide 
an understanding of the implications of adding additional capabilities to 
information flow reduction systems. Simplifying assumptions are made to 
enhance that understanding. In particular, the arrival process of information 
from the collection system is considered to have the following features as 
displayed in Figure 2: 

° Poisson departure process from the collection system 

° Arrival rate from the collection system = Acorect: 

Similarly, except as noted, each successive device i in the reduction 
system services each information item according to: 

° Exponential service time distribution 

° Service rate for device i = L.. 

Accordingly, each system device sees a Poisson arrival process in steady 
state [Ref. 1] and uses an exponential service time distribution to achieve its 
requirements. Together these assumptions provide a mathematically 


tractable basis to conduct the analysis of the flow reduction system itself. 
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Figure 2. Flow Reduction System Overview 


B. SYSTEM WITH NO FLOW REDUCTION 


Information overload occurs when the decision maker is presented 
with more information than he/she has the ability to handle. The decision 
maker may attempt to examine all of the information but the rate of flow to 
the decision maker is greater than his/her capacity for that flow. 

Define: 

Pcorecr 440 =.~—SsCs Ratio of arrival rate of information from the 
collection system to the service rate of the decision 


maker when analyzing the information. 


= hoortecr / Lp: 
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If Acourecr is greater than or equal to y1,,, information is arriving faster 
than the decision maker can process it. The items which have been collected 
pile up and the decision maker is not able to process them all. Also the 
decision maker is spending all of his/her time analyzing information and no 
time acting. A flow reduction system is required to control the arrival process 
and reduce the fraction of time the decision maker is busy analyZing 
information. 

The decision maker is perhaps the person best suited to judge the 
fraction of time, Pp, he/she should devote to analyzing information. The 
remaining time can be spent acting on that information. Since Pp, is fixed by 
the decision maker and [,,, is considered to be a fixed value for the decision 
maker, a flow reduction system is required if Pcortecr, defined to be Agourecr / 
Lp, is greater than or equal to 1.0. A flow reduction system is also desired if 


Pcottecr iS greater than pp, but less than 1.0. 
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ASSUMPTIONS IN MODEL DEVELOPMENT 


The following assumptions apply throughout model development: 


ue 


The decision maker has perfect understanding of the value of an 

item he/she has just analyzed. This means the decision maker 

cannot make Type I and Type II errors. 

The decision maker has historical information regarding the 

information environment and has been able to estimate its 

parameters. This implies the decision maker has an estimate for 

the probability that any item collected from the environment is 

Good or Bad. The decision maker also has an estimate on the 

distribution of trait levels of both Good and Bad items. 

The information environment remains constant throughout 

the duration of analysis. There is no change to the parameters of 

the items collected such as: 

e probability an item collected by the collection system is good 
or bad, 

e rate at which items arrive from the collection system, 

e or probability that a particular good or bad item has a 
particular trait level. 

No distinction is made in the service time requirements of a 

Good and a Bad item. Both items have the same service time 

distribution given they are being analyzed by the same device. 

All devices have room for all items arriving to that device, there 

is no loss due to lack of capacity. 

All devices service the arriving items on a First-Come, First- 

Served basis. 


Mechanical device i services items at rate HW, such that U; > 


AcoLLECT ; 


Ocottecr iS greater than ppv. 
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PD: NONDISCRIMINATING FLOW REDUCTION MODEL 


The first flow reduction model examined is of a system which provides 
a reduction of information to the decision maker without discriminating 
between the values of the information it is accepting or rejecting. Nor is a 
prioritizing system available to improve efficiency in the throughput of good 
information. The decision maker sees only the flow necessary to achieve Ppy. 
Figure 4 depicts such a process. This model produces base-case results by 
which enhanced models, which distinguish good items from bad, can be 


evaluated. 


Decision 


AA 4 adr 
\ AcOLLECT 
Unbiased sees 


Selection 


Fraction of Good = Po (1-a) ACOLLECT 


Fraction of Bad = Pp 





Figure 4. Nondiscriminating Flow Reduction System 


Allow a to be the fraction of the collected flow that is desired to reach 


the decision maker. Clearly, 


i, 


do ai p DM 


a= 
Acouect  Prouecr . 

Define a success as an item surviving the selection process and a failure 
as trashing an item. Perform the selection process independent of any trait of 
the object or any previous selection or trashing and set the probability of 
success to a. Therefore for each item the selection process is an independent 
Bernoulli trial with success probability poy / Peooitsct- 

The formulas for determining the measures of effectiveness for all 
models are derived in the Appendix. For an understanding of the process, 
some of the derivations for this simple model are presented here. The 


probability a good item will be selected is 


P{Value = Good, Selected} 


P{Value = Good | Selected} = 
P{Selected} 


J 


which is simplified by independence of the selection process to 


P{ Value = Good}- P{Selected} 
P{Selected} 
P{Value = Good} 
= Ie . 


P{Value = Good | Selected} 


li 


In a similar fashion, 


Pi Value ="Bad" "Selected — 1, 
P{Value = Good | Trashed} = P,, 
P{Value = Bad | Trashed} = P,. 


The probability of a Type I Error is the probability an item is trashed 


and it is a good item. It is also by definition the fraction of the total flow that 
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is good and is trashed. The probability of a Type II Error is the probability an 
item is selected and it is a bad item; this is also the fraction of the total flow 


that is bad and is selected. Thus 


P{Type I Error} = P{Value = Good | Trashed}. P{Trashed} = P, {! _—Ppm _ 


PCoLLECT 


P{Type II Error} = P{Value = Bad | Selected}. P{Selected} = P, (<2) 


PcoLLect 


This simple system is nondiscriminating in its selection process and 
serves as a yardstick for measuring systems that discriminate by selecting good 
items over bad. This means the nondiscriminating system’s Type I and Type 
II Error probabilities are the highest of any calibrated selection system. 

Because the selection process is nondiscriminating and requires no 
machine effort to examine each item, we assume the time to achieve 
selection per item is a fixed value D. Further we assume that D << 1/pu,,, due 
to the simplicity of the task. Therefore the delay by the selector is considered 
to be negligible compared to the rest of the system and is ignored. This results 


in an exponential departure distribution from the selector with rate 


Pp 


PcoLLect 


ADEPARTURE = -Acouzer = Aum = @ aAcotren 


which corresponds to a Poisson arrival process to the decision maker. 
Because the decision maker’s service time distribution is exponential, this 
system is a single exponential channel queue (M/M/1). If an item arrives to 
the decision maker when the decision maker is idle (not analyzing another 
item or making a decision), the decision maker will immediately begin 
analyzing the item. If the item arrives when the decision maker is busy the 


item is placed in a queue and will be analyzed by the decision maker on a 


lg 


First-Come, First-Served basis. Waiting time measures of effectiveness are 


easily obtained from an M/M/1 queueing system, [Ref. 1], by the equation 


l 


We Uservice RATE > AARRIVALRATE 


? 
ee RATE ~ AARRIVALRATE 
co, else. 


The measures of effectiveness for this model are displayed in Table 2 
through Table 4 with numerical results obtained using the example data 


provided in Table 1. 


EXAMPLE DATA FOR EVALUATION OF MOEs 
Parameter Equation Value 
Probability an item from the | 
environment is good. 
Probability an item from the 
environment is bad. 
Service rate of the decision maker. 10 Items/Hr 


Arrival rate from the collection system Meodteen 30 Items /Hr 
Ratio of the collection rate to the decision 

maker’s analysis rate. Impossible to 

achieve all analysis if greater than 1.0. 


Fraction of the decision maker’s time the 
decision maker spends analyzing. 


Table 1. Nondiscriminating Flow Reduction Model Example 
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PROBABILITY MEASURES OF EFFECTIVENESS 
MOE Equation 
Pom GOOD 


Probability an item is good _| _Ppm 


and it reaches the decision PcoLLEct 
| maker. 


Prype T ERROR 


Probability an item is good 
and it is trashed. 


Prrasts BAD 


Probability an item is bad 
and it is trashed. 


PryPE IT ERROR 


| Pom __ 


Probability an item is bad 
and it reaches the decision 
maker. 


PcoLLECT 





Table 2. Nondiscriminating Flow Reduction Model Probability MOEs 


ANALYSIS TIME MEASURES OF EFFECTIVENESS 
MOE Equation Example Result 


Ppm coop p 
DM GOOD 
Fraction of decision 
maker's analysis time 
ent on good items 


Pow GOOD Ls Prypent ERROR 


P-TYPE I ERROR 
PryPE II ERROR 


Fraction of decision 

maker's analysis time 

spent on bad items 
Table 3. Nondiscriminating Flow Reduction Model Analysis Time MOEs 


Pov coop + PeypE TI ERROR 





Z| 


WAITING TIME MEASURES OF EFFECTIVENESS 
MOE Oe = Example Result 


Wom GOOD 


Expected total time in 
the system for good 
items reaching the 
decision maker. 


Wrype If ERROR 


Expected total time in 
the system for bad items 
reaching the decision 
maker. 


WorrasH BAD 


Expected total time in 
the system for bad items 
which are trashed. 


Wore I ERROR 


Expected total time in 
the system for good 
items which are trashed. 
Table 4. Nondiscriminating Flow Reduction Model Waiting Time MOEs 





E, DISCRIMINATING FLOW REDUCTION MODEL - SINGLE TRAIT 


A trait-based selection process seeks to improve the quality of the 
information reaching the decision maker while still reducing the overall flow 
from Acouiscr tO Apy. To achieve the enhanced quality, the selectors 
discriminate to select items that have traits known to be more prevalent in 
good items. The result is a probabilistic technique for selection. Woosley, 
[Ref. 2], provides a description of a number of techniques which achieve this 
goal. The methods include: classical inference, Baysian inference, 
Dempster/Shafer inference, fuzzy set theory, cluster analysis, estimation 
theory and entropy. To utilize this method the decision maker must possess a 
sensor which has the ability to examine each item and determine the trait 
level associated with a particular trait of that item. The decision maker must 
also believe that the trait being analyzed will assist him in selecting good 
items over bad items. 

For example, the decision maker may be receiving information in the 
form of messages. Assume that historically, messages which are good tend to 
be longer than messages which are bad. The decision maker chooses to use a 
sensor which is capable of measuring the length of each message. The 
selector examines the result of that measurement, selects the longer messages 
and trashes the shorter ones. Note that the trait “message length” is common 
to all information items of the type “message”. We require each item to 
possess the trait the sensor is examining. 

To prepare for selection based upon trait, the decision maker examines 
historical records of information items of this type and separates them into 
two groups: items which are good and items which are bad. The sensor is 
used to examine the trait level of each item and probability densities for both 
the good items, fgoop(t), and the bad items, fp,p(t), over the trait levels are 
formed as demonstrated in Figure 5. Note the two densities overlap but 


display a separation which assists in differentiating good items from bad. 


23 


Note also that there is no requirement for a good (bad) item to be present at 


all trait levels, nor is there a requirement for the densities f,,,(t) and feoop(t) to 


be of the same type. In fact, the further the two densities are separated by trait 


level and skewness the more useful the trait is for use in the selection 


process. Completely separate densities would provide perfect classification of 


items. 


Probability 


TET eee 





PELE ee ek 


Figure 5. Trait Level Probability Densities By Item Value 


Define 
F = 


r 


LL = 


F 


p = 


Sensor for analyzing the single trait used for selection. 
Rate at which F determines trait levels for each item. The 
rate 1" is independent of the value of the item. 


Fraction of the time F is analyzing information. 


As in the nondiscriminating case, the time required to compare the 


measurement result from the sensor with Known selection cutoff criteria 1s 


considered to be negligible when compared with the sensor’s processing time 
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and the decision maker’s analysis time. It is therefore ignored in this model. 
Because F is a machine and capable of much higher speeds than the human 
decision maker, yt" > up, and by Assumption 7 (section C) pF > Acorecp OF 
equivalently 0 < p* < 1.0. 

This separation process provides three distinct areas of interest defined 
by the parameters of f,oop(t) and f,,,(t) as demonstrated in Figure 6. From 
trait level T, to trait level T,, Area I, the probability that an item is good is 
zero and the probability that an item is bad is positive. Therefore, items 
displaying a trait level t € [T), T,) should be trashed since they are bad with 
probability 1.0. Area III shows the probability that an item is good is positive 
while the probability that an item is bad zero. Therefore, any items displaying 
a trait level t e [T,, T,] should always be selected since they are good with 
probability 1.0. Areas I and II take advantage of the offset in the distributions. 
Area II, [T,, T,), has positive probabilities for both good and bad items. The 
decision maker must have a method to determine which items from Area I 
should be selected and which trashed. Here the advantages of skewed 
distributions becomes apparent. In this case it is obvious that choosing an 
item with a higher trait level will result in a greater probability that the item 
is good. Let S* € [T,, T,) be the trait level above which the selector will select 
the item and below which the selector will trash the item. A method for 
determining the appropriate S’ is discussed in Chapter IV. The resultant 


system is shown in Figure 7. 
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Figure 7. Single Trait Discriminating Flow Reduction System 
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Appendix A contains the derivations of the measures of effectiveness. 
Table 5 provides example data for MOE evaluation. The resultant measures 


of effectiveness are collected in Table 6 through Table 8. 


EXAMPLE DATA FOR EVALUATION OF MOEs 


ed e-veataicoe Equation Value 


Probability an item from the 
environment is good. 


Probability an item from the 





environment is bad. 


Service rate of the decision maker. Hom 10 Items/Hr 


Arrival Rate from the collection 


system. Acouect 30 Items/Hr 


Ratio of the collection rate to the 


decision maker’s analysis rate. Pcoiectr 


Impossible to achieve all analysis if 
greater than 1.0. 

Fraction of the decision maker’s 
time the decision maker spends 


analyzing. 


Lowest trait level for bad items. Tr, 0.0 
Lowest trait level for good items. T 0.1 
Highest trait level for bad items. FO 

ies 


T 
Highest trait level for good items. T 





o8 


No 


WwW 


Cutoff trait level. 





SF set to reduce flow to Apy s° 0.701 
Probability densities : good items. aN, Beta(t; 4, 2, 0.1, 1.1) 
Probability densities : bad items. face) Beta(t; 5, 5, 0.0, 1.0) 
Service rate for sensor F. ut 50 Items /Hr. 


Table 5. Single Trait Flow Reduction Model Example 


Recall that the beta density on the interval (A, B) is given by [Ref. 3] 


we pe (AY Gey <x<B 
Bea B-A 
0 








else 


Define Fooop(t) and F,,,(t) as the cumulative distribution functions 


(CDFs) of feoop(t) and fgap(t) respectively. 


PROBABILITY MEASURES OF EFFECTIVENESS 
MOE Equation Example Result 


Pom GOOD 
0.198 
Probability an item is good 
1 Sj Base-Case = 0.080 
and it reaches the decision (Base-Case ) 
maker. 
P typeI ERROR 


0.102 
Probability an item is good (Base-Case = 0.220) 
and it is trashed. : 
PrRASH BAD 


0.632 
Probability an item is bad (Base-Case = 0.513) 
and it is trashed. i 
PiyPE TERROR 


0.068 

Probability an item is bad 
= = 0.167 

and it reaches the decision (Base-Case = 0.187) 


maker.. 





Table 6. Single Trait Flow Reduction Model Probability MOEs 
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ANALYSIS TIME MEASURES OF EFFECTIVENESS 
MOE Equation Example Result 


Ppm Goop 


Fraction of the decision P wees 
maker’s analysis time Pov coop + Prype error (Base-Case = 0.30) 


spent on good items. 


P-type Il ERROR 


Fraction of the decision Eira 0.26 


maker’s analysis time Pom coop + Prypemerror (Base-Case = 0.70) 


spent on bad items. 





Table 7. Single Trait Flow Reduction Model Analysis Time MOEs 
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WAITING TIME MEASURES OF EFFECTIVENESS 
MOE Equation Example Result 


Wom GOOD 


Expected total time in 1 z 1 0.55 Urs. 


the system for good 1 eee. (Base-Case = 0.50) 
j items reaching the 


I decision maker. 
Wrype II ERROR 


Expected total time in 0.55 Hrs. 


the system for bad items (Base-Case = 0.50) 


reaching the decision 


maker. 


Wrrasu BAD 


0.05 Hrs. 
(Base-Case = 0.0) 


Expected total time in 
the system for bad items 
which are trashed. 
Wr I ERROR 
0.05 Hrs. 
(Base-Case = 0.0) 


Expected total time in 
the system for good 


items which are trashed. 





Table 8. Single Trait Flow Reduction Model Waiting Time MOEs 


The results from the example data, when compared with those from 
the base-case, demonstrate the power and cost associated with discriminating 
selection. With a single trait system the probabilities of retaining a good and 
trashing a bad item are improved by 150 and over 20 percent respectively 
while the probabilities of Type I and Type II errors decreased to 46 percent and 
36 percent of their original values. The fraction of the time the analyst is 


working on good information improved by 150 percent and the fraction of 
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time spent on bad items decreased to about a third of its original value. These 
improvements are not without a cost. The waiting time MOEs demonstrate 
that the trait based selection process induces a delay in the arrival of 
information. That delay is seen by all items entering the system and, in this 
instance, increases the system delay by 0.05 hours or a ten percent increase for 
selected items. It is therefore important for the decision maker to understand 
the implications of a discriminating selection enhancement when designing a 
flow reduction system. The sensitivity of the system to minor changes is 


discussed in Chapter IV. 


F. DISCRIMINATING FLOW REDUCTION MODEL - DUAL TRAIT 


A multiple trait flow reduction system is similar to the single trait 
system in its discriminating towards selecting good items over bad and its 
capability for reducing overall flow to the decision maker. These systems use 
a network of selectors to perform the same task as a single selector system but 
with the goal of further enhancing the quality of the information reaching 
the decision maker and reducing the Type I and Type II errors the system 
produces. 

As illustrated in Figure 8, the network of interest is two selectors in 


tandem. Other selector networks can be formed by extending this model. 
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Figure 8. Two Trait Flow Reduction System 


To perform multiple trait analysis we require a different sensor to 


measure each trait. Define: 


F Sensor associated with the first selector, 

uF Service rate for sensor F, 

pF Fraction of time the sensor F is analyzing an item, 
G Sensor associated with the second selector, 

The Service rate for sensor G, 

p° Fraction of time the sensor G is analyzing an item. 


Again, we require i! > Agoitecrp and 0 < p’< 1, for allie {F, G}. 
We also seek traits which exhibit conditional independence. This 
implies that knowing the value of an item and its trait level for the first trait 


does not help in predicting the trait level of the second trait or, 


P{F =x ! Value = v}-P{G=y | Value = v}=P{F=x, G=y! Value=v} 
V xyes 


SZ 


Therefore a reduction in the flow by any selector 1 preceding selector j does 
not change the probability distributions associated with trait /. 

Figure 9 depicts the two sets of trait level probability distributions as 
sampled by sensor F and sensor G respectively. The parameters are labeled 
similarly to the single trait model with the addition of a superscript to 
indicate the sensor which is measuring the item. Both traits display a 
separation across their respective trait levels, {F Area I, F Area Ill, G Area I, G 
Area II}, which can be exploited by the selection process. The probability 
density functions (PDFs) associated with the first trait are f.gop(t) and fgan(t) 
for good and bad items respectively. Their CDFs are F.o9p(t) and F,,,(t). 
Similarly for the second trait the PDFs and CDFs are g.oop(t), Span(t), Geoop(t) 
and Gzap(t). The cutoff trait level used by the first selector is S* and the cutoff 


trait level for the second selector is S°. 


Selector 1 Trait Distributions Selector 2 Trait Distributions 


F Area I F Area ll F Area Ill 


G Areal G Areall G Area III 














fgap(t) / 
BAD! / 


y 
di: 


TF Try SF TF, TF; TSgT SG teres 
Trait Level Sampled by Sensor F Trait Level Sampled by Sensor G 


Probability 
Probability 





o 


Figure 9. Two Trait Selection Model Probability Densities by Trait Level 


Sensor F is associated with the first selection process and performs its 
function in the same manner as the sensor in the single trait model. The 


output from Area I of sensor F’s densities plot, Figure 9, as well as that in F 
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Area II which is less than S' is still sent to the trash. The output from F Area 
Iil is routed directly to the decision maker, by-passing the second selector. 
However, in the two trait selection model the selected items from F Area II 
are no longer routed directly to the decision maker but instead proceed to the 
second selector. The flow rate into the second selector is therefore dependent 
upon the values T*, and S* as well as the distributions foop(t) and fg.n(t) for 


the first trait. This rate is calculated in Appendix A and is: 


AG = (P, : beers a) 7 Fooop (s*)] Canae ras ae a Feap (s*)]} ; Acouect ; 


To simplify follow-on equations define the probability a collected item 


is routed from F to G as 


Peto = (Po -[Faoon(Tz )- Facon (S*)]+Ps -[Faan (Tz )-Faan(S*)]) 


Using this notation we calculate the probability an arrival to the second 
selector is good or bad. Define these probabilities as Pgggp; prog ANd Prapi toc 


where, 


P = Poet ECTED GOOD F AREA II 
GOOD IF TOG P 
FTOG 2 
P = PseLECTED BAD F AREA TI 
BAD IF TOG P 
FTOG 


Except that the arrival flow rate is determined by the first selector and 
the probability that an arrival to the selector is good has changed, the second 
selector works exactly like the single trait selector previously modeled. 
Because the trait levels are conditionally independent given value, any 
adjustment in the cutoff setting, S¥ for the first selector does not impact the 


probability distributions for the second trait. 


Sample data for the parameters of the system are found in Table 9. The 
measures of effectiveness for the two trait model are presented in Table 10 


through Table 12. 
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EXAMPLE DATA FOR EVALUATION OF MOEs 
veveetacos _ Equation Value 


Probability an item from the 
environment is good. 

| Probability an item from the 
environment is bad. Ps 07 


Service rate of the decision maker. 10 Items / Hr. 


Arrival rate from the collection 
system. “eee 30 Items / Hr. 





Ratio of the collection rate to the 
decision maker’s analysis rate. Oeantcen 
Impossible to achieve all analysis if 
preater than 1.0. 
Fraction of the decision maker’s 
| time the decision maker spends 
analyzing. 


First Trait : Lowest trait level for 
| bad items. 


First Trait : Lowest trait level for 








ood items. 
| First Trait : Highest trait level for 


| bad items. T", 1.0 
| First Trait : Highest trait level for - 
good items. VT; 11 


First Trait: PDF good items. Beta(t; is 20.1, Ee 





fcoop 
First Trait : PDF bad items. ag Beta(t; 5, 5, 0.0; ag} 
Second Trait : Lowest trait level for - 
bad items. I" 0.0 
Second Trait : Lowest trait level for 
good items. 
Second Trait ; Highest trait level 
for bad items. 
Second Trait : Highest trait level 
for good items. 
ili 
2. _Frobebliy Scoontt) Beta(t; 5,5, 0.1, 1.1) 


Second Trait : Probability 
distribution bad items. Seavit) Beta(t; 2, 4, 0.0, 1.0) 


Service rate for sensor F Ul 50 Items / Hr. 


Table 9. Two Trait Flow Reduction Model Example 
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PROBABILITY MEASURES OF EFFECTIVENESS 


MOE 


Pom GOOD 


Probability an item is 
good and it reaches the 
decision maker. 


Pie I ERROR 


Probability an item is 
good and it is trashed. 


Pras BAD 


Probability an item is 
bad and it is trashed 


Piype TT ERROR 


Probability an item is 
bad and it reaches the 
decision maker. 


Equation 


P, -(1-Fooon (TE )) 


+ 


Pease ‘Pooop iF ToG -(1-Ggoon(S*)}} 


[Paze : Pooop 1 FTOG ecaso 


Rall) 
+ 


Peto ‘PEaDIFTOG -GanlS | 


Prtoc *Psapirtoc ‘(I=GaolS-)) 


Example Result 


0.214 


(Single Trait = 0.198) 
(Base-Case = 0.080) 


0.086 


(Single Trait = 0.102) 
(Base-Case = 0.220) 


0.648 


(Single Trait = 0.632) 
(Base-Case = 0.513) 


0.052 


(Single Trait = 0.068) 
(Base-Case = 0.187) 





Table 10. Two Trait Flow Reduction Model Probability MOEs 
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ANALYSIS TIME MEASURES OF EFFECTIVENESS 
MOE Equation Example Result 


Ppm coop 
0.805 
Fraction of the decision PoMcoon 


maker’s analysis time Pomcoop + Prypem ERROR (Single Trait = 0.74) 


| spent on good items. (Base-Case = 0.30) 


PTyPE I ERROR 
0.195 
Fraction of the decision i eee or 


maker’s analysis time Pow coop + Prype merror (Single Trait = 0.26) 


spent on bad items. (Base-Case = 0.70) 





Table 11. Two Trait Flow Reduction System Model Analysis Time MOEs 
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WAITING TIME MEASURES OF EFFECTIVENESS 
MOE Equation _ : _ Example Result 


1 0.571 Hrs. 


Expected total time in the 
system for good items 
reaching the decision maker. (Base-Case = 0.50) 


(Single Trait = 0.55) 


Wry 11 ERROR 0.574 Hrs. 


Expected total time in the 
system for bad items reaching 


the decision maker. 
me) eae) ea] 
ES ea | aS 
ee A coect Az Hou — Avm 


WrrasH BAD 0.053 Hrs. 


(Single Trait = 0.55) 
(Base-Case = 0.50) 


Expected total time in the 
system for bad items which 
are trashed. 


1 }: Peto -PaapiFtoc ‘Gap (s°) ( 1 
LW" — Acouecr eee P3apiFtoc ‘Gsan(S°)| +E Feap (S*) foc 


(Single Trait = 0.05) 
(Base-Case = 0.00) 


0.054 Hrs. 


Wry I ERROR 


Expected total time in the (Singlegie tae) 
system for good items which 
are trashed. 


( l }: Petros ‘PeoopiFT0G Geoenl oan { 1 
oe Aone = - Pooop i FT0G ‘ Ggoon(S*)] + Po =Eoceuls || we - Ae 


Table 12. Two Trait Flow Reduction Model Waiting Time MOEs 


(Base-Case = 0.00) 
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G. SORTING ENHANCED FLOW REDUCTION MODEL 


The single trait flow reduction model demonstrates the ability of a 
discriminating selection system to improve the quality of the information 
reaching the decision maker. The dual trait system improves the quality 
even further and allows a slight time savings for good items when compared 
with bad items in the same reduction system. This time savings is the result 
of items which are known to be good bypassing the second selector and 
proceeding directly to the decision maker’s in-box. Once in the in-box, these 
known good items must wait in a first-come, first-served queue behind items 
which may be good or bad. The sorting process is designed to reduced the 
waiting time of good items by giving priority within that in-box to items 
which have a greater probability of being good. Since the priority system does 
not reduce the number of bad items reaching the decision maker, it will not 
improve the overall quality of the flow. That is achieved only by the 
selectors. It will, however, allow the good items to reach the decision maker 
sooner. 

The sorting enhancement model replaces the single slot in-box used by 
the decision maker with a three level prioritized in-box. The model 
examines the benefits and costs associated with adding this in-box to a single 
trait flow reduction system. 

Define the three slots of the in-box as 1, 2, and 3. Let slot 1 be the slot 
that the decision maker accesses first when reaching for a new information 
item to analyze. If there are no items within slot 1 the decision maker obtains 
an item from slot 2. Should both slot 1 and slot 2 be empty the decision 
maker obtains an item from the third slot. If all slots are empty the decision 
maker obtains the next item arriving to any slot in the in-box immediately 
upon its arrival. If an item arrives while the decision maker is busy, it is 
placed into the appropriate slot and remains there to be serviced in first-come, 
first-served order. No arriving item can interrupt a busy decision maker who 


is analyzing another item, even if the new item holds a higher priority. 
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The in-box and decision maker combination described is a non- 
preemptive single channel queueing system with a Poisson arrival process 
and an exponential service rate distribution. The waiting times in queue 


associated with each slot ie {1, 2, 3} of the in-box are calculated using [Ref. 4], 


GH 
w = fal \ Hy 


7 (1-6; )(1-9;) 


where 
if = total number of slots (3), 
Ay. = the arrival rate to slot k € {1, 2, 3}, 
Ly = the service rate for items of the type placed in slot k, 
= Up for all slots, 
= = : - ica— Oo} 
jai Bj 


To obtain the total waiting time associated with the in-box slot as well as the 
decision maker, the expected service time per item, 1/[p,y, is added to this 
value. 

The arrival rate, A,, to each slot i is determined by how the selectors 
feed the in-box. Because the objective is to analyze good information, items 
will be placed by priority in the in-box with the items that have the highest 
probability of being good items in the highest slot. Items of lower probability 
will fill lower slots. From the single trait flow reduction model we know that 
items in Area III, Figure 10, are good with probability 1.0. Clearly, we should 
forward all items with trait levels in the interval (T,, T,] to slot 1 of the 
priority in-box. Define S* as the cutoff trait level for the selection process of 
this single selector. The remaining items that have been selected to reach the 
decision maker have trait levels on the interval [S‘, T,]. We must define a 
method of dividing this remaining flow into the two remaining in-box slots. 
As in the single trait model, the skewness of the densities f.gop(t) and f,,p(t) 


demonstrate that items with higher trait levels tend to have a greater 
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probability of being good items. Define S, as a cutoff trait level such that 
items on the interval [S,, T,] are placed in slot 2 of the in-box and items on the 
interval [S*, S,) are placed in slot 3, where S’< S, < T,. Figure 11 depicts this 


selection and prioritizing process. 
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Figure 10. Single Trait Flow Reduction Model with Prioritized In-box Cutoffs 
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Figure 11. Single Trait Flow Reduction with Prioritized In-Box Model 


Table 5 provides the example data for the single trait model. Table 13 
adds the additional information required for MOE evaluation. The 
probability and analysis time MOEs do not change from the single trait model 
and are located in Table 6 and Table 7 respectively. The waiting time MOEs 


are presented in Table 14. 


EXAMPLE DATA FOR EVALUATION OF MOEs 


Parameter Equation 





Cutoff trait level for In-Box. 
S, set to minimize Wpxy coop 


Table 13. Priority In-Box Single Trait Flow Reduction Model Example 
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WAITING TIME MEASURES OF EFFECTIVENESS 


MOE 
Wp coop 


Expected total time in the 


system for good items 


reaching the decision maker. 


re Tl] ERROR 
Expected total time in the 
system for bad items 
reaching the decision maker. 


Wrrasu BAD 


Expected total time in the 
system for bad items which 


are trashed. 
Wrye I ERROR 


Expected total time in the 
system for good items which 


are trashed. 


|SXepsEtateya 


Provided in 


Appendix A. 


Provided in 


Appendix A. 


Example Result 
0. 466 Hrs. 


(Single Trait = 0.55) 
(Base-Case = 0.50) 


0: 793 binS 


(Single Trait = 0.55) 
(Base-Case = 0.50) 


0. 05 Hrs. 


(Single Trait = 0.05) 
(Base-Case = 0.0) 


0.05 Hrs. 


((Single Trait = 0.05) 
Buse-Cuse = U.0) 





Table 14. Priority In-Box Single Trait Flow Reduction Model 


Waiting Time MOEs 


IV. CALIBRATION OF SELECTORS AND SYSTEM SENSITIVITY 


A. CALIBRATION OF SELECTOR CUTOFFS 


il Setting the Single Trait Selector to Achieve the Desired Flow 
Reduction 
SF is the trait level for the single trait selector system above which all 
items are routed to the decision maker. Recall, 


do a Ew ; AcotLect 


PcotLect f 


and that the items above S* are both good and bad with CDFs Fooop(t) and 





Fzap(t) respectively. Therefore, the fraction of flow allowed to reach the 
decision maker must be Ppy / Poortect- 

Select S* such that, 

—— = (P, -[1-Fooon(S*)]+ Ps -[1- Foo(S"))) 

Note that to achieve Ap, only a single SF setting is available as 
illustrated in Figure 12. The lower graph plots the resultant flow to the 
decision maker against the setting S’. S" can assume values from T"*, (0.1) to 
T*, (1.0). By setting S* to T’, the only flow removed from the system is the 
small fraction of known bad items with trait levels less than T’,. Setting S* to 
T*, removes all flow from the system except for the known good items which 
have trait levels above T*,. Because the decision maker in the example has 
the capacity to analyze a total flow of 10 items per hour but also desires to use 
only 80 percent of his/her total time analyzing items, the desired flow to the 
decision maker is 8 items per hour. An S! setting of 0.701 is therefore 
required. The upper graph in Figure 12 plots the fraction of the items that 
reach the decision maker that are good, Ppy coop, against S*. Note that this 
value increases by setting S* to a higher trait level. However, a higher S'* 


setting results in a corresponding decrease in flow to the decision maker. 
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This tradeoff must be understood by the decision maker when calibrating the 
selector and when determining the fraction of available time to be used 


analyzing information. 
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Figure 12. Single Trait Selection Setting to Achieve Flow Reduction and the 
Corresponding Quality Flow to the Decision Maker 


ze Two Trait Selector Settings to Achieve Flow Reduction and 
Maximize Quality 

The dual trait model has two selector settings, SF and S°, which must be 
set to reach the desired reduction in total flow. Unlike the single selector 
model, however, there are an infinite number of setting combinations which 
achieve this reduction. If S* is a setting which allows more flow than the 
desired Ap, to leave the first selector, S° can be set to decrease that flow. As 
demonstrated in the single selector model and in Figure 12, if S" is set to 0.701 
the flow departing the first selector is reduced to Apy. The second selector, 


however, will still trash all items with trait level below T°,, Figure 9. 
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Therefore, the flow reaching the decision maker is less than the desired flow 
of Apy. S* must be set lower than T", to account for this reduction. Clearly, 
the setting S° is uniquely determined once S* has been set. The upper graph 
in Figure 13 plots S° against S* denoting all valid combinations of the settings 
which reduce Acorecr tO Ap. Recalling that the decision maker desires the 
highest quality of information possible given the available settings, the lower 
graph in Figure 13 plots the quality achieved, ppy coop, versus S’. A 
maximum value of 0.805 occurs when SF equals 0.635. The S®° setting which 


corresponds to an S* of 0.635 is 0.3783. 
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Figure 13. Dual Trait Selector Settings to Achieve Flow Reduction 


and Maximize Quality 
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B. CALIBRATION OF A PRIORITY IN-BOX 


The priority in-box model uses the single trait selector model as its 
basis. Apy is achieved by setting S‘ to 0.701. All items leaving the selector 
with trait levels above T’, are placed in slot 1 of the in-box and have the 
highest priority, Figure 10. The remaining flow has trait levels between S* 
and T',. These items must be placed in either slot 2 or slot 3 as discussed in 
the model’s development. The cutoff setting S, is the setting above which 
items are placed into slot 2 and below which items are placed into slot 3. The 
role of the prioritized in-box is to reduce the expected waiting time in the 
system for good items by reducing the expected waiting time in the decision 
maker’s in-box. Therefore, S, is selected to provide the lowest expected 
waiting time for good items. Figure 14 plots the expected waiting time for 
good and bad items versus S,. The solid line is the waiting time for good 
items and has aminimum value of 0.466 hours when §, is set to 0.77. Note 
that this is well below the result of 0.55 hours for the single selector model 
without a prioritized in-box. The expected waiting times for bad items are 
plotted with a dashed line. Note that for all settings of S, the bad items take 
longer to pass through the system than the good items. When 5, is set to a 
level close to the lower bound of slot 3 or the lower bound of slot 1 the system 
behaves similarly to a two slot in-box. The separation of the curves at the 
boundaries results from the priority of slot 1 and that all items in slot 1 are 
good. At optimum this time separation allows good items to reach the 
decision maker 40 percent faster than bad items. Therefore there is never a 


disadvantage to prioritizing the flow to the decision maker. 
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Wait for Good and Bad Items 
Single Selector Without Prioritized In-Box 
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Figure 14. Single Selector with Prioritized In-Box Model 


Waiting Times versus S, 
€ SYSTEM SENSITIVITY ANALYSIS 


1. Two Trait Selector Sensitivity to Acoirecr 

It is important to understand the implications of not recalibrating a 
flow reduction system when changes occur external to the system. Figure 15 
displays 8 quality curves plotted against S". The dark curve provides the 
decision maker with 8 items per hour as discussed in model development 
and calibration. The lighter curves correspond to amounts from 10 to 3 items 
per hour as indicated. 10 items per hour is the maximum rate of flow a 
decision maker can analyze with the given service rate of up,, = 10. A higher 
rate causes information overload. Also at A,,, = 10 the decision maker is 
spending all of his/her available time analyzing items (pp, = 1.0) and no time 
making decisions. For the example system, S* cannot be set to a level which 
allows for 2 or fewer items per hour. Note that, if the model setting for S* 


(0.635 as indicated by a vertical line) is retained and the arrival rate to the 
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decision maker declines due to a decrease in A, oiscy, there is an increase in 
the quality of arriving items. This may occur when the rate at which items 
are collected decreases but the environment from which the items are 
obtained does not change. Also note that though the quality is higher, the 
setting does not provide the highest quality possible for that rate of arrivals. 
In this case S‘ should be reset to achieve optimum quality. A similar decline 
in quality occurs when A,,, increases through an increase in Agoiyecr and a new 
optimum quality is reached by decreasing S*. Therefore, the actual collection 
rate must be monitored and the system recalibrated to maintain optimum 


quality levels. 
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Figure 15. Recalibration Requirement Due to Changes in Acoyect 


Pde Model Sensitivity to Changes in the Probability a Collected Item 
is Good 
Changes in the information environment as well as changes in the 


collection system may induce a change in the probability that a collected item 
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is good, P,. This change in P, is directly reflected in the quality of information 
reaching the decision maker as measured by Ppy coop. Figure 16 demonstrates 
this effect. The figure plots Pp coop against P, for both the single and dual 
trait selector systems. Note that over the entire range of P, the dual trait 
system provides higher quality information to the decision maker and both 
systems are better than the non-discriminating base-case. If a system 
produced lower quality than base-case (the light gray area) it would not be 
used. Recall that the example data calibrated the system based upon an 
estimated P, value of 0.30. Indicator lines are plotted at P, = 0.2 and P, = 0.4 to 
indicate a region of uncertainty in the actual value of P,. The resultant 
Ppm coop Value levels are also plotted. For the single trait selector system, this 
uncertainty results in Ppy coop Values ranging from 0.5 to 0.725. The same 
uncertainty imposed upon the dual trait model results in Ppy coop values 
from about 0.7 to 0.86. Therefore the dual trait system is much less sensitive 


to fluctuations in P, in the estimated region. 
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Figure 16. Impact of Changes in P, on Ppy coop for 


Single and Dual Trait Systems 


The same curves as in Figure 16 are shown in Figure 17 to demonstrate 
that they can be used to evaluate the impact of adding an additional selector 
to the system or adjusting the collection system or the information 
environment to achieve higher levels of information quality. In this case we 
assume the decision maker has the ability to adjust the collection system 
and/or in some fashion change the environment to increase P,. It is assumed 
that these additions or changes result in a cost to the decision maker which 
can be estimated. Using the example data and a single trait system, a Pp coop 
value of about 0.74 is achieved at a P, of 0.3 (see Figure 17). Suppose the 
decision maker desires this quality to be increased to 0.8. The addition of a 
second selection achieves this value directly. If a selector is not added to the 
system, the decision maker must change the collection system and/or the 


information environment such that the new P, value is approximately 0.5. 
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The costs associated with each change can be compared and the most cost 


effective option chosen. 
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a ie 7. Adding a Second Selector or Adjusting P, 


The priority in-box is sensitive to changes in P, as illustrated in Figure 
18. As in Figure 14, the expected time in the system for both good and bad 
items is plotted against S,. The four sets of curves show the expected delay if 
the system receives information from the environment at Pz, levels of 0.1, 0.3, 
0.5 and 0.7 respectively. The system is assumed to have been recalibrated 
following the change in P,. Note that as P, decreases the delay in the system 
decreases for both good and bad items. The delay for good items approaches 
that of a non-priority in-box as P, approaches 1.0. Therefore there is no 
disadvantage across P, level to adding a priority in-box to a flow reduction 


system but a great advantage if P, is small. 
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Figure 18. Sensitivity of a Priority In-Box System to Changes in P, 
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V. CONCLUSIONS 


This thesis develops several analytical models for the evaluation of an 
information flow reduction system based upon _ probabilistic inputs. 
Sensitivity analyses demonstrate the requirement to monitor the estimated 
parameters of the system in order to maintain flow to the decision maker 
which is both acceptable and optimal. The study closes with some comments 


on the significance of the models and suggestion for further research. 


A. MODEL INSIGHTS AND USEFULNESS 


The four models presented provide an insight into the capabilities and 
limitations of probabilistic based flow reduction systems. Such systems can 
provide high quality information to the decision maker at the desired flow 
rate if traits can be found which provide separation and/or skewness between 
the value probability curves. Such systems require historical records to 
evaluate the information environment and are subject to fluctuations in the 
parameters of that environment. This implies two important concepts: 

° an understanding of the enemy’s methods and _ tactics is 
fundamental to operating these systems and, 

° the enemy can take advantage of friendly use of these systems by 
changing their operating parameters or hiding truly vital information in low 
probability regions. 

Selector-based systems can increase flow quality with a trade off in 
delay to the operator. The models utilize machine-based sensors to sample 
the trait level of an item. Therefore, in most cases, the delay is not significant 
when compared with the human decision maker. This may not always be the 
case and the decision maker must take into account the service rates of all 


“devices” in the reduction system, some of which may be human as well. 
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The priority in-box demonstrates the ability of the system to decrease 
the delay of good items reaching the decision maker. This decrease occurs 
across every calibration setting. Therefore there is no disadvantage to 
implementing a prioritizing system. In the models the prioritizing system 
was placed just before the decision maker but this does not need to be the case. 
A prioritized in-box can be used following every selector in the reduction 


system and increase the speed of good items across the entire network. 


B. RECOMMENDATIONS FOR FURTHER STUDY 


Further study is warranted in two distinct areas. The first area deals 
with the operation of the system to maintain optimal conditions. The second 
focuses on managers of these systems and anticipated developments. 

In the development of this thesis an assumption was made which 
stated that the parameters of the information environment did not change 
and the system was allowed to attain steady state. However, analysis revealed 
that these systems are susceptible to fluctuations in those parameters and 
need recalibration if the parameters change. A procedure should be 
developed to monitor the changes in the information environment to 
include Apo ecy Pe feoop{t) and fgap(t) (for each monitored trait). 

Managers of such systems must anticipate changes in the requirements 
associated with the system. Procedures should be developed to answer 
management questions such as: 

° If an investment is to be made, what portion of the system 


should be improved/replaced? 


° Is the output quality of the system as high as the system is 
capable of achieving? How do we know? 

° What is the limiting device in the system? 

° Can this system achieve my desires? 

° Is the system doing what I expect? How do we know? 
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° Do we have the best quality/quantity mix? 

e If we have to remove a device, which should it be? 

° Are the selectors in the optimum order? 

Tactical questions involve the exploitation of such systems. How can 
the enemy exploit our use of these systems? How can we exploit the enemy’s 
use of them? What information about these systems is vital to suppress from 
the enemy and thus what information is important for us to obtain about 


theirs? 
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APPENDIX. DERIVATIONS OF MEASURE OF EFFECTIVENESS 


iL. NONDISCRIMINATING SELECTION PROCESS 


A. 


P DM GOOD 
Pom coop 18 the probability an item is good and it reaches the decision 


maker. The probability any item reaches the decision maker is 


Ao _ _ Pom 


Acousct Protect - 


Ci 


The probability an item is good given that it is selected is 


P{Value = Good, Selected} 


P{Value = Good | Selected} = 
P{Selected} } 


and by independence, 


P{Value = Good}-P{Selected} 
P{Selected} 
P{Value = Good} 
= P,. 


P{Value = Good | Selected} 


Therefore, Poy coop equals 


Poacoop = P{Value = Good | Selected}. P{Selected} 


=P, (2 
PcoLtect 


D2 


B. Pipe I ERROR 
Pye eRROR 1S the probability that an item is good and it is trashed. The 
probability any item is trashed is 


\a=[1-—Pos_) 
Pcottecr /: 


The probability an item is good given that it is trashed is 


P{Value = Good, Trashed} 


P{Value = Good | Trashed} = 
P{Trashed} 1 


and by independence, 


P{Value = Good}- P{Trashed} 
P{ Trashed } 
P{ Value = Good} 
= ee 


P{Value = Good | Trashed} 


Therefore, Pryper error equals 


Pvocterron = P{ Value = Good || Trashed}-P) Prasiied? 


= Pe : t = Ppm } 
Protect 
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Cc. Porras BAD 
Prrasy Bap 1S the probability that an item is good and it is trashed. 


Therefore, Prpasu gap equals 


Prrasupap = P{Walue = Bad | Trashed}-P{Trashed} 


D. PryPE I ERROR 
Prype n error 1S the probability that an item is bad and it is selected. 


Therefore, Pyyr 7 error equals 


PryPeneRRor = Fy Value = Bad || Selected}-P{Selected? 
-», {fo} 
PcoLLEcr 


ES Ppm coop 


Ppm coop is the fraction of the decision maker’s time spent on good 
items. This is the fraction of items that reach the decision maker that are 
good. The fraction of all item that are good and reach the decision maker is 
defined to be Pp coop. The fraction of all items that are bad and reach the 
decision maker is defined to be Pre n error) Therefore, the fraction of those 


items that reach the decision maker that are good is 


fe DM GOOD 


Pom GOOD Bg Prype H ERROR . 
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|e P+yPE II] ERROR 


The same derivation is used to determine Pyype 7 ERROR, Which becomes, 


P. TYPE IT ERROR 


Pom coop + PryPE 1 ERROR . 


G. Wom coop 21d Wrype 1 error 

Items that reach the decision maker have two opportunities to be 
delayed. The first delay is from the selection process and the second delay is 
waiting for and then being analyzed by the decision maker. The selector delay 
is D and is very small when compared to Hp,,. Therefore the only delay is that 
seen at the decision maker. This is the delay for a single channel M/M/1 


queueing system and the result is 


1 


How — Ape . 


H. Werrasy sap 2Nd Wrype 1 error 


Items that do not reach the decision maker have one opportunity to be 
delayed; the selection process. The selector delay is D and is very small when 


compared to Up, and is considered to be zero. 
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Pay SINGLE TRAIT ANALYSIS WITHOUT PRIORITY IN-BOX 


A. Pom GOOD 
Pym coop 18 the probability an item is good and it reaches the decision 
maker. Good items reach the decision maker only if they possess trait levels 


greater than S’. Therefore, Pp, coop equals 


Piucoop = P{Selected | Good!-P{Good} 
T; 
=a. WiZoon yes 
sf 


SAP Be | eteceon( Sat 


B. Pir I ERROR 


Prypet error 1S the probability that an item is good and it is trashed. Al] 


items with trait values below S* are trashed. Therefore, Prypr1 error equals 


Prypererror = P{Trashed | Good}-P{Good} 
sf 


Pg ° eon oa 


T, 


=P, si eree Sn). 


C. PorasH BAD 


Prrasy Bap iS the probability that an item is good and it is trashed. All 


items with trait values below S' are trashed. Therefore, Pras gap equals 


Preastpap = P{Trashed | Bad}-P{Bad} 
gf 
=) | foav(s)ds 
To 


=P one en 
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D. Piypr Il ERROR 


Prype n ERROR 1S the probability that an item is bad and it is selected. Bad 
items reach the decision maker only if they possess trait levels greater than S. 


Therefore, Prypr 1 error equals 


Pomcoop = P{Selected | Bad}. P{Bad} 
T, 
= ane | faan(s)ds 
gf 


SP el ee nlm) 


E. Ppm coop 


Ppm coop 1S the fraction of the decision maker’s analysis time spent on 
good items. This is the fraction of items that reach the decision maker that 
are good. The fraction of all items that are good and reach the decision maker 
is defined to be Phy coop. The fraction of all items that are bad and reach the 
decision maker is defined to be Prey error. Therefore, the fraction of items 


which reach the decision maker that are good is 


Pow GOOD 


Pow GOOD + Prype 1 ERROR . 


F. Prypr II ERROR 


The same derivation is used to determine Prype 7 ERROR Which becomes, 


P TYPE II ERROR 


Pow GOOD 1 PLypE ERROR : 
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Gis Woiesenand ia aeeeeas 

Items that reach the decision maker have two opportunities to be 
delayed. The first delay is from the selection process and the second delay is 
waiting for and then being analyzed by the decision maker. The selector delay 
is exponential with rate 1". The decision maker delay is also exponential with 


rate Upy. Both delays form single channel M/M/1 queueing systems and the 


result is 
1 1 
a 
ML" —Acouscr pm ~ 4pm 
H. Wrraso pap 2Nd Wye: ERROR 


Items that do not reach the decision maker have one opportunity to be 
delayed; the selection process. The selector delay is exponential with rate p". 


This delay forms a single channel M/M/1 queueing system and the result is 


I 


Lp — Ap 
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> TWO TRAIT ANALYSIS 


A. Pom GOOD 


Pom coop 1S the probability an item is good and it reaches the decision 
maker. In the two trait selector system, good items can reach the decision 
maker directly from either selector. Good items reach the decision maker 
directly from the first selector only if they possess trait levels greater than T.’. 


This value is 


=P, [Fost Ge (1 - Feoop{T,")) 


Good items reach the decision maker from the second selector only if 
they are selected by the first selector but not sent directly to the decision maker 
and then are selected by the second selector. Define P,,,, as the probability an 


item is sent from the first selector to the second selector. Then 


PEtoc = Ie, ‘|Beoon (T;)—Fooop (s*)| ae ‘[Foap (Tr )—Feap (s*)]) 


Using this notation we calculate the probability an arrival to the second 
selector is good or bad. Define these probabilities as Pagop; prog ANd Paani ertoc 


where, 


P SELECTED GOOD F AREA II 


FG [Fret 
(Pc -[Feoon (Tz ) - Faooo(S a 


FG: | Eeaep es Foon (SF ) 
)-F 


Feat eae 


Pooop 1FTOG = Pp 
FTOG 


Hear IG om 


and 
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P = PSELECTED BAD F AREA II 
BAD! FTOG P 


FTOG 


Py ° [faa | 
© (Py -[Fooon (TE) = Feoon(S*)]+ Pa “[Foan (TF) ~ Faan(S*))) 


(P, eae Oe Feoop (s*)} + Ps i eae (T; ) —Feap (s*))) 


For all arrivals to the second selector it acts like the single selector in the 


single trait case. Therefore, P,., coop equals 
[Po . (1 — Feoop OG )) a baa G- Peoop I\FTOG © (1 = Goon (s# ))} 


B. Paver IERROR 


Pier Error 1S the probability that an item is good and it is trashed. In the 
first selector, all items with trait values below S'* are trashed. This probability 


was Calculated in the single trait case to be 
Po Feoon(S") 


For the second selector the items must arrive to the selector, be good items 
and be trashed. Items with trait values less than S° are trashed, therefore the 


total value is 


(Pe “Foo00(S" )|+| Penge -Pooop iF TOG “Goon lSale 
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Cc. PrRasH BAD 


Pras pap 1S the probability that an item is good and it is trashed. In the 
first selector all items with trait values below S' are trashed and in the second 
selector all items with trait levels below S®° are trashed. Therefore, Press pap 


equals 


IP, -Fyan(S")|+[Prroc ‘Psaptetoc *Gpap (s°)}. 


D. Payer Il ERROR 


Pir 1 ERROR IS the probability that an item is bad and it is selected. Bad 
items reach the decision maker only if they possess trait levels greater than S* 
in the first selector, are sent to the second selector, and have trait levels 


greater than S®° in the second selector. Therefore, Pre 1 error equals 
Eero ‘ PEaDIFTOG err (s°))}. 


E. PpmM Goop 


Opm coop is the fraction of the decision maker’s time spent on good 
items. This is the fraction of items that reach the decision maker that are 
good. The fraction of all items that are good and reach the decision maker is 
defined to be Pp, coop: The fraction of all items that are bad and reach the 
decision maker is defined to be Prpen error) Therefore, the fraction of items 


which reach the decision maker that are good is 


E DM GOOD 


P DM GOOD + Prype 1 ERROR . 
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f Ptype Il ERROR 


The same derivation is used to determine Pyype 7 error, Which becomes, 


P. TYPE I! ERROR 


Pom GOOD + PryPE TERROR : 


G. Wom GOOD 

Items that reach the decision maker have three opportunities to be 
delayed. The first delay is from the first selection process and all items must 
pass this process. The second delay is for the second selection process. Only 
those items that have trait levels above S* but below T, are sent to the second 
selector. The remaining items, if they are good, are sent directly to the 
decision maker and do not see this delay. Once reaching the decision maker, 
all good items must wait for and then be analyzed by the decision maker. The 
first selector service time is distributed exponential with rate u*. The second 
selector service time is distributed exponential with rate n°. Only a fraction of 
items that are collected reach the second selector. This fraction is Pro, as 


previously calculated. Define 


F 
XG = Peroc  Acouect 


The decision maker's service time is distributed exponential with rate Up,,. 
The flow reaching the decision maker has been reduced to Apy. All devices 
form a single channel M/M/1 queueing systems. The delay added by the 
second selector must be weighted to include only the good items that pass 


through it. This weighting is 


Petoc ‘Pooopir toc I o\Ceees (s°)) 
Pg (1 - Feoop (TZ))+[Prroc *Peoop i FT0c -(I- GooD (s° ) 
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Therefore Wp, coop 1S 





1 Perog* Pootsirz *(1— Geoan(S")) I ! -( 


MW — Acousct Pe (leeaeo ie) | Perea heeemereee = aseu(S2))| He — Ae 


H. Wriyer lI ERROR 


All bad items that reach the decision maker must pass both selectors 
and wait to be analyzed by the decision maker. Therefore their waiting time 


1S 


] l l 
| 
BP —Acorrsct tad ——} 


L Wrrasit BAD 


Bad items are trashed by both the first and second selectors. Only a 
fraction of the bad trashed items are trashed in the second selector. Therefore, 
the waiting time associated with the second selector is weighted accordingly 


with the equation 


Pero Paanieroc “Gnan(S*) 
pero PaapietoG ‘Gyan (S°)|+ re oF eo om) : 


Therefore, Wrpasi pap 1S 





( ] }: Pe toc * Paapirtoc Ca { 1 
ye ~ Acowect = ‘Paani Ftoc ‘Gran(S° )|+ Ps Eon ue - Ac : 
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J. Wrver IT ERROR 


Good items are trashed by both the first and second selector. The 
fraction of the waiting time associated with the second selector is weighted to 


account only for those items that reach the second selector. This weight is 


Peto - Pooop i FTG “Secon Sa) 
peetee ‘Pooop FT0G ‘Ggoon(S*)| + Po “Fen . 


Therefore, W-rypr 1 error iS 


( l }: Petoc ‘Peoopirtoc * Gcoop (Ss?) ( l 
pe — Acouect pecs -Pgoop1 FT0G Gens) || + Pe “Pecan Gel ue ~AG . 


| 


4. SINGLE TRAIT ANALYSIS WITH PRIORITY IN-BOX 


A.  Non-Waiting Time Measures of Effectiveness 
All non-waiting time MOEs are quality oriented and are accounted for 
by the selection process. These MOEs do not change from the single trait 


selection derivations in Section 2 of this Appendix. 


B. Wom GOOD 


Good items that reach the decision maker wait at the first selector and 


the decision maker. The wait for the first selector is 


peer ee 
Hp — Am 


Good items that reach the decision maker are placed in any of the three slots 
in the priority in-box. The wait in any of these slots is found with the 


equation, [Ref. 4], 


5 i 
wi) = R= Uy 


7 (1-6,.,)(1-4;) 


where 
r = total number of slots (3), 
A, = the arrival rate to slot k e {1, 2, 3}, 
Ly. = the service rate for items of the type placed in slot k, 
= Lom for all slots, 
O, = ; Ai oa) 
j= B; 


There is an additional wait for service completion for all items of 1/Up,. 


Therefore, for each slot we determine the arrival rate and the probability that 


Ve 


a good item that arrives to the in-box enters that slot. These probabilities are 
used to weight the expected value of waiting time. Define the slots in order of 
priority as {1, 2, 3} and their corresponding arrival rates and slot entrance 
probabilities to be {A,, A,, 43} and {Pigoop, Pacoop, Pacoop} respectively. Recall 
that all items entering slot 1 are good and have trait values above T,*. Also, 
the items entering slots 2 and 3 are both good and bad with trait values 


between SF and T,' and a trait level cutoff between slot 2 and 3 of S.. 


Therefore, 
A, = Foon fas )): ‘Acouect’ 
A, = ([P G (F Peel F oop (S, ))/+{ P,° (1- is BAD\ (S .))) Acouect: 
i. a, =([Po-( F oop (S;) Bepook (s*)) )) i |Ps-{ (Feap S:) F pap (S* ))}  Acouect! 

FG ease = ) 
TO) = | aT 
eS Feooo(5"))) 

p z [Pe sees & \-F OD coop (Si )) 

2GOOD P,-( P,. (1 Exoon a . 

P Z [ee en) Pesos) 

3GO0D ~~ (P| = (i- Foon (S (s*)) 


For simplicity define the total arrival rate as 
Ay =A, tA, +43. 
Therefore the expected waiting time for good items in the in-box is 


1 2 3 
Wooopinsox = Picoop Wy + Fycoop Wy?) + Pscoop “Wy”. 


q 2 if 


is, 


And the expected total wait in the system for good items that reach the 


decision maker is 


l l 
Wo GOOD — (| aT WcoopinBox i (+) | 


C Wrypr Il ERROR 


The expected waiting time for bad items that reach the decision maker 


is calculated in the same manner. Define the probability weights as 


Ea (I= Fean(S,))] 


Py pap = P, (= Fyan(S*))) 


and 


ee [Py -(1-Feao(S*)) 


Therefore the expected waiting time for bad items in the in-box is 


= (2) (3) 
W 5 aDINBOX am Py pap aN an Py pap “Wi ; 


And the expected total wait in the system for bad items that reach the decision 


maker is 


1 1 
WeypE HERROR = (——j}-—— + Weapingox + | 
ue =A conc i Hom /. 
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D. Werrasn pap 2Nd Wrype : error 


Good and bad items that are trashed only wait in the first selector. That 


wait is 


Cele : 
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